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Abstract: In this paper, the implementation of algorithm 
proposed in ( Nzabanita , J., et al. 2012) for some known linear 
structures on the covariance matrix 27 is performed and 
simulations for different sample sizes are repeated many 
times . For these simulations, the percentages of non positive 
definite estimates are produced, and the linear structures are 
identified and classified . 

Keywords: Growth curve model, Estimator, Linearly 
structured covariance matrix, Positive definite matrix. 

1. Introduction 

A growth curve is an empirical model of the evolution of a 
quantity over time. Growth curves are widely used in biology 
for quantities such as population size, body height or biomass 
(Pan, et al., 2002). In mathematical statistics, growth curves are 
often modelled as being continuous stochastic processes, e.g. as 
being sample paths that almost surely solve stochastic 
differential equations (Seber, G. A. F. and Wild, C. J., 1989). 
The growth curve model has an important application in many 
areas such as medicine, pharmacy, natural sciences, social 
sciences, etc. The growth curve model has been extensively 
studied over many years and it was introduced in (Pothoff, R. 
and Roy, S., 1964). With improvement the growth curve model 
was extended and studied by many authors, for instance 
(Nzabanita, J., et al., 2012; Seid Hamid, J. and von Rosen, D., 
2006; Verbyla, A. and Venables, W., 1988; Yokoyama, T., 
1996; Srivastava, M. S. and von Rosen, D., 1999). 

The estimation of parameters in the growth curve model, 
when the covariance matrix has some specific linear structure 
has been discussed by some authors, for example (Nzabanita, J., 
et al., 2012; Ohlson, M. and von Rosen, D., 2010). In Ohlson, 
M. and von Rosen, D. (2010), when the classical growth curve 
model with linearly structured covariance matrix is considered, 
a suggested estimation procedure gives explicit and consistent 
estimators of both the mean and the covariance matrix, and in 
(Nzabanita, J., et al.,2012), when the extended growth curve 
model with two terms and a linearly structured covariance 
matrix is studied, also a suggested estimation procedure gives 
explicit and consistent estimators of the covariance matrix. 

The idea is first to estimate the covariance matrix when finding 
the inner product in a regression space and thereafter re- 
estimate it when it should be interpreted as a covariance matrix. 
This idea was first considered by (Ohlson, M. and von Rosen, 
D., 2010) and is exploited by decomposing the residual space, 
the orthogonal complement to the design space, into orthogonal 
subspaces. Studying residuals obtained from projections of 


observations on these subspaces yields explicit consistent 
estimator of the covariance matrix. However, through 
simulation for some linear structures on the extended growth 
curve model, it was noted in (Nzabanita, J., et al. 2012) that 
the estimates of the covariance matrix may not be positive 
definite for small sample sizes whereas it is always positive 
definite for some other structures for moderate sample sizes. 
Hence, in this paper we studied how the problem of non- 
positive definiteness for the estimates of the covariance 
matrix 27 for some linearly structured covariance matrices 
would be identified. 

The model we study may be defined in the following way: 

Let X ! p x n be an observation matrix, A i : p x q. be a 

within individual design matrix, B i : x be a parameter 

matrix, C- :k i xn be a between individual design matrix, for 

i=l, 2, r(C 1 ) + p<n and E(C 2 )czE(C i) where r(-) and 

<2(*) represent the rank and the column space of a matrix, 

respectively. The extended growth curve model with two 
terms and a linearly structured covariance matrix is defined as 
follows, 

X — A^BfDy + A^Br^fZ^ + E ( 1 ) 

where the columns of E are assumed to be independently 
distributed as a p-variate normal distribution with mean zero 

and a positive definite covariance matrix 27 =(cr ^ ^ i.e., 
E~MN p J0,Z,I n ). 

The covariance matrix 27 has some linear structure. The 
matrices A t and C, are known matrices whereas matrices Bj 
and 27 are unknown parameter matrices. 

2. Estimators in the growth curve model with a 
linearly structured covariance matrix 

In this section, we derived estimators for the covariance 
matrix in the extended Growth Curve model with a linearly 
structured covariance matrix. 

2.1. Linearly structured matrix and linear structures 

Definition 2.1 (Linearly structured matrix) A matrix 

27= cr-- is linearly structured if the only linear structure 

u 

between the elements is given by cr- = cr^ and there exists at 
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least one (i,j)i^(k,l) so that I cr- I = I cr^ I . 

The linear structures for the covariance matrices emerged 
naturally in statistical applications and they are in the statistical 
literature for some years ago. More details on the examples of 
structures are developed in subsection 2.2. 

2.2. Different linear structures for the covariance 
matrix 

4- Covariance matrix with zeros 

An example of a covariance matrix with zeros is given by 


(°\ 
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4- Banded covariance structure 

The banded covariance structure can for example be 
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4- Toeplitz and circular Toeplitz covariance structure 

Toeplitz covariance structure is given by 
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and the circular Toeplitz covariance structure is given by 


Her = 


/ a 2 
pi a 
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pi p 2 pi 
2 Pi P2 

p2 pi O 1 pi 

\pi P'2 Pi <* 2 ) 
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4- Intraclass covariance structure 

The intraclass covariance structure looks like 
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4- Compound symmetry (type I and II) 

Votaw, D. F. (1948) extended the intraclass model to a model 
with blocks called compound symmetry, type I and II and are 
follows 
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2.3. Estimators in an extended growth curve model 

Considering the extended growth curve model given by 
equation (1), X=A 1 B 1 C 1 +A 2 B 2 C 2 +E, but with E~MN p>n ( 0,lf s) ,I n ) 
where lf s) is a linearly structured covariance matrix. Assuming 
that matrices A h C, for i=l, 2 are of full rank and that 


E(A 1 )nE(A 2 )={0}. The main estimator of the linearly 
structured covariance matrix £ (s) proposed by (Nzabanita, J., 

et al. 2012) equals 

vecS< s > = T + ((T + )'i>'<i>T + J (T + )'$'vecS. (2) 

where S is the sum of squares matrix given by 
S = H 1 H' 1 +H 2 H , 2+H3H3 311(1 

a = (n - ri )I + (rj - r 2 )f 1 <s»f 1 + r 2 T 2 ©T 2 with 

= 1 - P a,,£< s > - P f lA2 J< s > and T 1 = 1 - P Al i' s) where ® 

denotes the Kronecker product and vecM denotes the 
vectorization of M. T + is the Moore-Penrose inverse of T and 
T is a matrix such that v£cE (s) (K)=Tv£c£ (s) where v£c£ (s) (K) 

is a columnwise vectorized form of 2 . Hj , H 2 and H 3 are 


projectors given by H 1= X(I - P, ) , H 2 = a - P )X(P C . - P C2 ) 
and H 3 = a - P |S , - P |A2:? , )XP C2 . 

The above projectors are obtained from the whole space 
decomposition according to the within and between 
individual designs illustrating the mean and residual spaces, 
see Figure 1 . 

X{ s) and Z 2 s) are ^e estimators of X (s) obtained by 

/V /\ 

considering only the residual and by considering both 
and H 2 respectively (see Figure 1) and are given by 

vecZ ls) = - 2 — T + ((T + )'T + ) -1 (T + )'vecS 1 . (3) 

1 n — r 1 

and 

vec£ 2 s) = T + ((T + ) Y YT + ) _1 (T + ) Y vec(fl 1 fl 1 + fl 2 fl 2 ). (4) 
where y = (n - r x )I + (r 3 - r,)?!®?,. 

The estimators developed above have some properties like 
unbiasedness and consistency, these properties should be 
proved else refers to the paper where in (Nzabanita, J., et al. 
2012), it has been shown that 


The estimator Z[ s) given 

in 

(3) is 

a 

consistent 

estimator of ^ , i.e., ) 

p 

->2 (s) . 



A / \ 

The estimator Z 2 given 

in 

(4) is 

a 

consistent 

estimator of ^ , i.e., Z 2 

p 

->S (S) . 



The estimator Z (s) given 

in 

(2) is 

a 

consistent 

estimator of ^ , i.e., 

p 

->E (S) . 





Figure 1 : Decomposition of the whole space according to the 
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within and between individual designs illustrating the mean and 
residual spaces where V l = (? Z(J) (A { ) ,V 2 = (2^ {s) (Tj A 2 ) , 


_L 


v 3 =(e 2 , ( , ) (A 1 )+e z(f) (T 1 A 2 )) M=£( c 2 ), 

v 2 =(e(c’ 1 )ne(c’ 2 )) 


_L 


w 3 =<% CO, Mi=P A ^ (J) XP c; , 




P (J) XP where ± denotes the orthogonal complement. 


3. 


Simulation Studies 


3.1. Description of Scenarios 

Simulations were done with Matlab code implementing the 
formula (5) for every linear structure discussed in section 2. In 
each simulation a sample of observations was randomly 
generated from a p-variate growth curve model. Sample sizes 
from n=10, 20,..., (small), n= 100 (moderate) to n=500(\arge 
sample size) were considered. The simulations are repeated 
N=1000 times and the following design matrices were used 
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for p = 4 and 
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for p = 5 and 


r 


Q = 


v 


ones(l, nl 2) 
zeros (l,n/2) 


J 


zero s (l, n I 2)^ 
ones( 1, nl 2) 

C 2 = (zeros(l,n!2\ ones(l,n/2)) where n is the sample 

size, At and C t are the within and between individual design 
matrices respectively for i=l, 2. 


3.2. 


Results and discussions 


For every structure, the averaged estimates of the covariance 
matrix for small sample size n = 10 is reported, the graphs of 
the percentages of non-positive definite estimates are plotted, a 
graph and a table that summarize all information are reported. 

4- For covariance matrix with zeros 

When the covariance matrix with zeros is considered, the 
averaged estimate of 
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for n- 10 is given by 
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E = 


/ 257.0860 
0 

- 88.0042 
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0 \ 
- 33.2569 
2.7536 
64.7439 


For small sample size n = 10 the averaged estimate is not 
closed to the proposed value of E. 
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300 


Figure 2: Percentage of non-positive estimates of E for 
covariance matrix with zeros 

4- Banded covariance structure (with p = 4) 

The averaged estimate of the positive definite covariance 
matrix 

0 
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E = 

for n= 10 is given by 

/ 77.8090 
- 15.6526 


17 

1 

0 

V° 


1 

4 

2 

0 


o 

3 


o\ 

0 

3 


E = 


V 


0 

0 


- 15.6525 

54.3274 

86.7390 

0 


0 

86.7390 

105.2802 

- 16.9658 


0 \ 
0 

- 16.9658 
114.3014 ) 


For small sample size n = 10 the averaged estimate is not 
closed to the proposed value of E. 
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Figure 3: Percentage of non-positive estimates of E for 
banded covariance structure with p=4 

4- Toeplitz covariance structure (with the same 
variances) 

For this linear structure the estimate of the positive definite 
covariance matrix 

/4 1 2 3 \ 
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for ft = 10 is given by 

f 4.4566 1.3752 2.1237 3.308l\ 

1.3752 4.4566 1.3752 2.1237 

2.1237 1.3752 4.4566 1.3752 

3.3081 2.1237 1.3752 4.4566/ 


The obtained averaged estimate of E is closed to the true value. 



Sample size n 

Figure 4: Percentage of non-positive estimates of E for toeplitz 
covariance structure with same variances 



4- Circular Toeplitz covariance structure (with p = 5) 

When this structure is considered, the estimate of 
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Here the averaged estimate of E is closed to the true value 
and this structure shows a zero percentage of non-positive 
definite estimates of E for all sample size n (see Table 1). 


4- Intraclass covariance structure 


When this structure is considered, the estimate for n =10 of 
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The averaged estimate of E is closed to the true value and this 
linear structure shows zero percentage of non-positive 
definite estimates of E for all sample size n (see Table 1). 


4- Toeplitz covariance structure (with different variances) 

For this structure, the estimate of the positive definite 
covariance matrix 
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for ft = 10 is given by 

/ 123.9509 
- _ 0.1276 

E - -28.6401 

\ 83.8243 

For small sample size n = 10 the averaged estimate of E is not 
closed to the proposed value of E. 

45 f- 


CO 

CD 

O 


40 - 


£ 35 


CD 

"O 

CD 30 
> 

CD 

* 25 


20 - 


CD 

CO 

crs 

tz 

ID 

O 


15 


£ io 


JH 


50 


100 


150 


250 


300 


200 

Sample size n 

Figure 5 : Percentage of non-positive estimates of E for toeplitz 
covariance structure with different variances 


4h Circular Toeplitz covariance structure (with p = 4) 

When this structure is considered, the estimate for n =10 of the 
positive definite covariance matrix 
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The averaged estimate of E is closed to the true value. 


4- Compound symmetric type I structure 

When this structure is considered, the estimate for n =10 of 
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In this case, the averaged estimate of E is closed to the true 
value but this structure shows only a small percentage of non- 
positive definite estimates of the covariance matrix E for 
sample size around n = 10 (see Table 1). 


4- Compound symmetric type II structure 

For this structure the estimate of 

/5 2 3 i\ 

2 5 13 
^ “ 3 16 4 

^1 3 4 6y 


for ft = lOis given by 

/ 37.0450 -3.9938 -26.6162 

^ _ -3.9938 37.0450 19.2260 
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19.2260 \ 
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For this linear structure the averaged estimate is not closed to 
the proposed value of E for small sample size ft = 10. This 
structure shows a small percentage of non-positive definite 
matrices of the estimate of E for small sample size around n 
= 10 (see Table 1). 

In summary, as said by (Nzabanita, J., et al. 2012), our results 
concluded that for some linear structures, the estimates of E 
may be positive definite or not for small sample size. The 
class of circular Toeplitz covariance, and intraclass 
covariance structures show 100% of positive definite 
estimates of E for all sample sizes, and compound symmetry 
(type I&II), the covariance matrix with zeros, the banded 
covariance and Toeplitz covariance (with the same/different 
variances) structures show a non-zero percentage of non- 
positive definite estimates of E for small and/or moderate 
sample sizes. 
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The Table 1 below shows the percentage of non-positive 
definite estimates of the linearly structured covariance matrix H 
for different sample sizes ( n ) and different linear structures (LS) 
for the extended growth curve model (EGCM) where LSI 
stands for the covariance matrix with zeros, LS2 for the banded4. 
covariance structure, LS3 for the Toeplitz covariance structure 
with the same variances, LS4 for the Toeplitz covariance 
structure with the different variances, LS5 for the circular 
Toeplitz covariance structure with p - 4, LS6 for the circular 
Toeplitz covariance structure with p - 5, LS7 for intraclass 
covariance structure (or uniform covariance structure), and LS8 
and LS9 for the compound symmetry structure type I and II 
respectively. The figure 6 shows the classification of the linear 
structures. 
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0 
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0 
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0 
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0.1 

0 

0 

0 

0 

0 

0 
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0 

0 

0.1 

0 

0 

0 

0 

0 

0 
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0 

0 

0 

0 

0 

0 

0 

0 

0 

290 

0 

0 

0 

0 

0 

0 

0 

0 

0 

300 

0 

0 

0.1 

0 

0 

0 

0 

0 
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Table 1: Percentage of non-positive definite estimates of H for 
different sample sizes n and different linear structures for 
EGCM. 

The linear structures are classified according to the graph of 
Figure 6 below 



Figure 6: Different covariance structures. (E wz = with zeros , 
£ (m) = banded , E T = Toeplitz , £ C t =circular Toeplitz , ^cs= 
compound symmetry and L IC = intraclass ) 

Concluding remarks 

In this paper, we implemented the algorithm proposed by 
(Nzabanita, J., et al. 2012). We identified and classified the 
structures that produce positive definite estimates for the 
linearly structured covariance matrix X where the class of 
circular Toeplitz covariance and intraclass covariance 
structures show 100% of positive definite estimates of H for 
all sample sizes. 
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